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We  show,  using  small  examples,  that  two  algorithms 
previously  published  for  the  Bilevel  Linear  Programming  problem 
(BLP)  may  fail  to  find  the  optimal  solution  and  thus  must  be 
considered  to  be  heuristics.  A  proof  is  given  that  solving  BLP 
problems  is  NP-hard,  which  makes  it  unlikely  that  there  is  a  good 
exact  algorithm. 


Bilevel  Linear  Programming  (BLP)  is  a  nested  optimization 
model  involving  two  problems,  an  upper  one  and  a  lower  one;  both 
problems  have  to  be  optimized  given  a  jointly  dependent  set 
s=((x>y)-0:  Ax+By<b}.  The  upper  decision  maker,  who  has  control 
over  x,  makes  his  decision  first,  hence  fixing  x  before  the  lower 
decision  maker  selects  y.  The  general  form  of  BLP  can  be  defined 
as: 

MAX  Cj  -x  +  dj  -y 
x 

where  y  solves: 

MAX  d2  -y 

(1) 

such  that: 

Ax  +  By  <  b 
x,  y  >  0. 
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In  this  paper,  which  is  a  part  of  a  broader  research  on  BLP 
[ Ben-Ayed  1988],  we  study  two  algorithms:  the  Parametric 
Complementary  Pivot  Algorithm  [ Bialas-Karwan-Shaw  1980,  and 
Bialas-Karwan  1984]  and  the  Grid  Search  Algorithm  [Bard  1983];  we 
show  that  those  algorithms  do  not  always  find  the  optimal 
solution,  and  we  point  out  some  of  their  potential  pitfalls. 
Finally,  we  prove  that  the  problem  of  solving  BLP  is  NP-hard; 
this  a  special  case  of  a  little-known  result  in  Jeroslow  [1985], 
with  a  simpler  proof.  The  NP-hardness  of  BLP  suggests  that,  as 
with  integer  programming  problems  (which  are  also  NP-hard) , 
algorithms  involving  some  form  of  branching  [ Falk  1973,  Gallo  and 
Ulklicu  1977,  Fortuny-Amat  and  McCarl  1981,  Bialas  and  Karwan 
1982,  Papavassilopoulos  1982,  Candler  and  Townsley  1982,  Bard  and 
Falk  1982,  Bard  and  Moore  1987]  are  to  be  preferred. 

1.  The  Parametric  Complementary  Pivot  Algorithm  (PCP) 

The  Parametric  Complementary  Pivot  Algorithm  (PCP)  [Bialas- 
Karwan-Shaw  1980,  and  Bialas-Karwan  1984]  is  distinguished  by  its 
popularity  and  the  large  number  of  papers  that  refer  to  it.  Most 
published  BLP  algorithms  compare  their  efficiency  to  that  of  PCP. 

When  replacing  the  lower  problem  in  (1)  by  its  Kuhn-Tucker 
conditions  after  introducing  dual  variables  u,  slack  variables  z 
and  surplus  variables  t,  an  equivalent  formulation  of  the  BLP 
problem  can  be  obtained: 
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MAX   Cj  -x    +    dj  • y 

such   that: 

Ax    +    By   +    z   =   b 

BTu    -    t   =   d2  (2) 

Yit,     =0 

uiZi     =0 

x,    y,    u,    z ,    t    >    0 . 

The  PCP  algorithm  uses  formulation  (2).  At  each  iteration, 
the  algorithm  tries  to  find  a  feasible  solution  that  gives  an 
objective  function  value  a  to  the  BLP  problem  by  solving  the 
following  system: 

Ax    +    By    +    z    =   b 

eIy    +    BTu-t   =   d2 

cx  •  x    +    dj-y    -    s    =    a 

(3) 
Yit4    =0 

uiZi     =0 

x,    y,    u,    z,    t,     s    >    0 

where  s  is  a  one-dimensional  surplus  variable,  I  is  the  identity 
matrix  and  e  is  a  positive  scalar  sufficiently  small  so  that  the 
solution  to  the  above  system  is  the  same  as  when  z    equals  zero. 
In  attempting  to  solve  (3),  Bialas  et  al.  added  the  positive 
definite  matrix  e I  to  use  a  technique  similar  to  that  proposed  by 
Wolfe  [1959]  in  solving  a  system  corresponding  to  convex 
quadratic  programming  problems. 
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Although  the  PCP  algorithm  may  find  the  optimal  solution  for 
some  BLP  problems,  this  is  not  guaranteed.  The  following  is  an 
example  for  which  PCP  does  not  give  the  optimal  solution: 

MAX  1.5xa  +  6yi  +  y2 

where  yx  and  y2  solve: 

MAX  Yl  +  5y2 

such  that: 

Xj  +  3yj  +  y2  <  5 

2Xj    +  y,  +  3y2  <  5 

x,  <  1 

xx  ,  yl  ,    y2  >  0. 

We  will  consider  the  problem  of  finding  a  solution  with 
upper  objective  value  a  >  2.  The  system  of  equations 
corresponding  to  (3)  is: 

X,  +  3y:  +  y2  +  z,  =  5 

2x,  +  y!  +  3y2  +  z2  =5 

Xj  +  z3  =  1 

.  01ya  +  3u2  +  u2  -  tj  =1 

.  01y2  +  Uj  +  3u2  -  t2  =5 

1 .  5Xj  +  6yx  +  y2  -  s  =  2 

yitj  =  y2  t2  =  u,z,  =  u2  z2  =0. 

The  PCP  algorithm  initializes  by  solving  the  LP  obtained  by 
ignoring  the  lower  objective  function.  In  this  example,  that 
gives  X!=y2=0,  ya =1.667.  The  complementary  slackness  conditions 


5 
then    require    that   ta =u2 =0    and  ux=.328.    The    fifth   constraint    above 
is   not    satisfied,    so   we    introduce    an   artificial   variable   w    such 
that    . OlVi +ux +3u2 -t2 +w   =    5.    This    gives    the    system: 

Yj     +     .333X!     +     .333y2     +     .333zx  =    1.667 

z2     +    1.667xj     +    2.667y2     -     .333z!  =    3.333 

z3     +    l.OXj  =    1 

u1    -    .OOlx,     -    .001y2    +    .333u2     -    .001za     -    .333tj  =    .328 

w    +    .OOlXj     +    .011y2     +2.667u2     +    .OOlz,     +.333tj     -    1 . 0t2  =   4.672 

s    +    . 5Xj     +    1.0y2     +    2-OZi  =   8. 

The  algorithm  performs  pivoting  operations  on  the  above 
system  in  order  to  make  w=0  while  preserving  the  complementarity 
conditions.  From  the  w-equation  above,  we  see  that  entering  xa  , 
y2  ,  u2  ,  Zj  or  tj  would  decrease  w.  However,  u2  cannot  enter 
because  z2=3.333>0.  Similarly,  zx  and  tx     cannot  enter.  If  we 
choose  y2  as  the  entering  variable,  z2  leaves.  At  the  next  step, 
we  may  have  u2  enter  (u:  leaves),  then  zx  enters  to  produce  the 
system: 

ya  +  .147x,  -  .059z2  -  .176s  =  .059 

y2  +  . 618Xj  +  .353z2  +  .059s  =  1.647 

z3  +  l.OXj  =  1 

u2  -  .OOlXi  +  3.0us  +  .001z2  -  l.Otj  +  .002s  =  .999 

w  -  .002xa  -  8Uj  -  .005z2  +  3 . Otj  -  1 . 0t2  -  .006s  =  1.985 

Zj  -  .059X!  -  .176z2  +  .471s  =  3.176. 
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The  only  variable  which  would  decrease  w  at  this  stage  is 
t,  ,  which  cannot  enter  because  y^O.  Thus,  the  PCP  algorithm 
stops  at  this  point  with  the  conclusion  that  a  solution  to  the 
problem  with  upper  objective  function  value  greater  than  or  equal 
to  2  cannot  be  found.  However,  x1=y2=l,  yi  =0  is  such  a  solution. 

In  this  small  example,  one  could  guess  the  optimal  solution 
using  hindsight.  For  example,  if  we  temporarily  allowed  w  to 
increase,  we  could  have  Xj  enter  and  yx  leave  in  our  last  system, 
which  would  then  allow  ta  to  enter  and  give  the  desired  solution. 
Also,  we  would  have  found  the  solution  if  x:  entered  instead  of 
y2  at  the  beginning.  However  this  example  is  sufficient  to  show 
that  the  PCP  approach  is  flawed.  On  larger  problems,  such  "quick 
fixes"  may  not  be  available. 

Bialas-Karwan-Shaw  [1980]  proposed  for  their  algorithm  a 
proof  based  on  techniques  similar  to  those  used  to  prove  Theorem 
3  in  Wolfe  [1959].  However,  the  two  situations  are  not  identical. 
In  particular,  condition  (e)  in  Bialas-Karwan-Shaw  cannot  be 
obtained  in  the  same  way  as  the  corresponding  condition  in 
Wolfe's  paper,  and  this  makes  the  proof  invalid.  A  specific 
counter-example  is  available  from  the  authors  and  is  also 
included  in  Ben-Ayed  [1988]. 

2.  The  Grid  Search  Algorithm  (GSA) 

The  Grid  Search  Algorithm  (GSA)  was  proposed  by  Bard  [1983]. 
The  author  claimed  that,  for  some  x*    between  0  and  1,  the 
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solution  to  a  BLP  problem  is  the  same  as  the  solution  to  the 
following  parameterized  LP: 

MAX  t*  (c,  -x  +  d,  -y)  +  (1  -  t*  )d2  -y 

x,y 

such  that : 
Ax  +  By  <  b 
x,  y  >  0. 

In  other  words,  by  finding  the  value  of  t*  ,  one  can  solve  BLP  as 
an  equivalent  LP.  Unfortunately,  the  statement  is  not  always 
true.  For  instance,  there  is  no  parameterized  LP  that  gives  the 
same  optimal  solution  as  the  following  BLP  problem: 

MAX  x  +  y 
x 

where  y  solves: 

MAX  -y 

such  that: 

4x  +  3y  >  19 

x  +  2y  <  11 

3x  +  y  <  13 

x,  y  >  0. 

The  GSA,  intended  to  find  x*  ,  starts  with  the  infeasible  solution 
(3,4)  when  t=l  (it  is  infeasible  because  substituting  x  by  3  and 
solving  the  lower  problem  would  give  a  value  for  y  that  is 
different  from  4).  3/5  is  the  only  value  of  t,  between  0  and  1, 
that  preserves  the  optimality  of  (3,4).  The  vertex  (4,1)  obtained 
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with  the  new  t  is  feasible  and  is  supposed  to  be  the  optimum 
according  to  GSA.  However,  the  actual  optimal  solution  is  (1,5). 
In  general,  if  the  GSA  is  currently  at  the  point  (x.y)  such  that: 
d2y>d2y*  and  c1  x+  (dj  -d2  )y=Cy  x*  +  (da  -d2  )  y*  ,  then  the  algorithm  has 
no  way  to  go  to  the  optimal  vertex  (x*  , y*  ) . 

Problems  with  GSA  were  independently  found  by  F .  A.  Al- 
Khayyal,  and  P.  Marcotte . 

The  GSA  is  very  quick;  it  could  be  used  to  provide  a  lower 
bound  for  other  algorithms  such  as  those  based  on  the  branch  and 
bound  technique.  However,  this  algorithm  is  risky  for  two 
reasons.  First,  as  is  the  case  for  PCP ,  it  does  not  tell  whether 
the  solution  it  gives  is  global  or  local.  And  second,  it  does  not 
provide  intermediate  results  (improved  upper  and  lower  bounds); 
if  the  algorithm  is  terminated  before  the  stopping  rule  is  met, 
no  solution  will  be  given,  not  even  an  approximation. 

3.  The  BLP  Problem  is  NP-Hard 

The  Knapsack  Optimization  problem  can  be  defined  as  the 
problem  of  choosing  from  a  given  set  of  natural  numbers  (a:  ,  a2  , 
. . . ,  an  }  a  subset  that  adds  to  the  largest  value  not  exceeding  a 
given  natural  number  B.  It  is  well  known  that  this  problem  is  NP- 
hard  (see  for  instance  Garey  and  Johnson  1979).  We  now  show  that 
if  we  could  always  solve  BLP  quickly,  we  could  solve  Knapsack 
Optimization  problem  quickly. 
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One  way  to  formulate  the  Knapsack  Optimization  problem  is 


N 
MAX  I    a±xd 

i=l 

subject  to: 

(4) 
N 

I    a4  x±  <  6 
i=l 

x,  =  0  or  1 . 


The  requirement  that  x±  equal  0  or  1  can  be  enforced  indirectly 
by  allowing  xd  to  be  any  real  between  0  and  1  and  making  the 
distance  from  x±     to  the  nearest  integer  as  small  as  possible. 
That  is,  the  constraints: 

x±  =  0  or  1 

can  be  replaced  by  the  requirement  that  x±  be  an  optimal  solution 
to  the  problem: 

N 
MIN  I    y4 
i=l 

subject  to: 

Yi  =  MIN  [x±  ,     (1-x,  )} 

0  <  Xi  <  1. 

Therefore  the  Knapsack  Optimization  problem  (4)  can  be 
reformulated  as  the  BLP : 
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N  N 

MAX  I    a±  x±  -  M  E  y± 
i=l        i=l 

where  the  y±  solve: 

N 
MAX   I  y± 
i=l 

such  that:  (5) 

N 

I    a±  Xi  <  B 
i=l 

y4  <  1  -  x± 
Xi  <  1 

Xi  ,  Yi  >  0 

where  M  is  a  large  number  to  make  the  minimum  of  the  sum  of  the 
y± s  equal  to  zero. 

The  following  result  shows  that,  if  M  is  chosen  sufficiently 
large,  every  non-integer  x  used  to  produce  a  feasible  solution  to 
the  BLP  (5)  is  inferior  to  an  integer  solution  z.  Since  there  are 
only  finitely  many  feasible  integer  solutions,  this  implies  the 
optimal  solution  to  the  (5)  is  integer. 

For  technical  reasons,  we  will  assume  that  MAX { a4  j  >  2.  We 
can  do  this  since  a  Knapsack  problem  with  all  a± =1  is  trivial. 

Theorem 

Let  M  >  (MAX{aA  ))2,  f(x)=  I    ax x, ,    g(x)=  I    MIN[x±  , l-x±  j .  If  x 
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is  feasible,  and  not  all  x±  are  integer,  then  there  is  a  feasible 
z  with  all  zA  integer  and: 

f(x)  -  Mg(x)  <  f(z)  -  Mg(z)  (6) 

Proof 

Let  Q  =  l-l/MAX{ai  j.  Our  assumption  implies  that  Q  >  .5.  We 
modify  the  feasible  x  in  a  sequence  of  steps  of  three  kinds: 

(1)  If  0  <  Xj  <  Q  for  some  j=l,  make  xi     -    0. 

(2)  If  Q  <  Xj  ,  Xj,  <  1  for  some  j  +  k,  replace  them  by  zi  ,  zk 
so  that  a3  Zj  +akzk=aj  xi  +  akxk  ,  Zj  +zk  >xi  +xk  ,  and  one  of  the  two 
new  values  is  1  while  the  other  is  between  0  and  1. 

(3)  If  Q  <  Xj  <  1  for  some  j,  with  all  other  components 
integer,  make  x.,  =  1. 

Each  of  these  steps  increases  the  number  of  integer 
components  of  x,  so  we  terminate  with  all  components  integer. 

If  we  let  z  be  obtained  from  x  by  a  single  use  of  step  (1), 
f(z)  >  f(x)-(MAX{ai  })Xj  .  If  Xj  <  .5,  g(z)  =  g(x)-xd  .  If  .5  <  X,  < 
Q,  g(z)  =  g(x)+x3-l  <  g(x)-l/MAX{ai  }  .  In  either  case,  (6)  holds. 

If  z  is  obtained  using  step  (3),  we  clearly  have  f(z)  >  f(x) 
and  g(z)  <  g(x),  so  (6)  is  immediate. 

If  z  is  obtained  using  step  (2)  we  have  f(z)=f(x).  The 
requirement  Zj +zk  >  x^ +xk  >  1  implies  that  g(z)  <  g(x)  except  in 
the  special  case  in  which  aj =ak  and  zt   ,  zk  >  .5,  in  which  case 
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g(z)=g(x).  However,  we  cannot  obtain  a  z  with  all  components 
integer  by  using  only  steps  of  this  kind,  since  each  such  step 
leaves  at  least  one  component  of  x  strictly  between  .5  and  1. 

Thus,  when  we  obtain  z  with  all  components  integer,  (6)  will 
be  satisfied.  It  remains  to  show  that  the  final  z  is  feasible,  in 
particular  that  Z  a±  x±  <  B  . 

Steps  (1)  and  (2)  clearly  preserve  feasibility.  To  show  that 
step  (3)  also  does,  note  that  Z  a±  z±  <  1  +  Z  a± xA  <  1+B.  Since  B 
and  Z  aA Zi  are  integer,  Z  a± z±  <  B .  Q.E.D. 

This  result  leaves  little  hope  that  a  polynomial  algorithm 
can  be  found  for  BLP,  and  suggests  that  the  situation  for  BLP  is 
similar  to  that  for  integer  programming.  In  fact,  BLP  can  be 
solved  as  a  mixed  integer  programming  problem  [ Fortuny-Amat  and 
McCarl  1982],  which  makes  it  an  NP-complete  problem. 


Acknowledgement:  The  authors  were  introduced  to  BLP  by  David  E. 
Boyce,  who  applied  Bilevel  Programming  to  the  study  of 
Transportation  Network  Design  problems  [Boyce  1986  and  LeBlanc- 
Boyce  1986] . 
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